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Problem  Statement: 

Effective  remediation  requires  that  educators  assess  student  variables  that  are  linked  to 
performance  deficits.  However,  extant  assessments  fall  short  in  providing  adequate  diagnostic 
information  about  how  medical  students  apply  their  knowledge  and  skills  to  authentic  clinical 
tasks.1  Still  further,  very  few  instruments  are  available  that  specifically  examine  student 
motivation  and  self-regulated  learning  (SRL)  processes  as  they  engage  in  relevant  and  authentic 
clinical  activities.  Because  research  has  shown  that  motivation  and  SRL  skills  are  malleable 
processes  and  predictive  of  success,  devising  ways  to  assess  these  processes  could  provide  a 
foundation  for  effective  remediation  and  instructional  supports.2 

Approach: 

Based  on  a  process  account  of  SRL,3  we  have  refined  an  established  assessment  methodology, 
called  SRL  microanalysis,2  and  used  it  to  evaluate  how  students  approach  (forethought),  monitor 
(perfonnance),  and  reflect  (self-reflection)  during  clinical  activities.4'5  Specifically,  as  students 
work  through  the  history,  physical  exam,  and  diagnosis  components  of  a  virtual-patient  case,  we 
use  SRL  microanalytic  measures  to  generate  qualitative  and  quantitative  data  about  students’ 
motivation  beliefs  and  SRL  processes. 

Lessons  Learned: 

Preliminary  results  suggest  that  SRL  microanalytic  measures  not  only  relate  to  short-  and  longer- 
term  outcomes,4  but  they  also  can  be  used  to  document  shifts  in  students’  motivational  beliefs 
and  SRL  during  clinical  reasoning  activities.5  Further,  recent  efforts  to  integrate  SRL 
microanalysis  with  virtual  patients  have  revealed  several  challenges  and  exciting  possibilities 
regarding  the  use  of  technology-integrated  assessments  as  part  of  an  existing  curriculum. 

Significance: 

This  assessment  methodology  is  innovative  because  it  obtains  real-time  data  on  students’ 
motivational  beliefs,  strategic  approaches,  and  adaptations  to  performing  clinical  tasks. 
Moreover,  this  approach  can  be  integrated  with  existing  virtual-patient  simulations.  Taken 
together,  SRL  microanalytic  data  can  be  used  to  track  student  progress  on  clinical  tasks,  thereby 
potentially  helping  to  improve  educators’  understanding  of  the  types  of  feedback  needed  to 
refine  and  optimize  student  motivation  and  SRL. 
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Assessing  Contextualized,  Dynamic  Processes: 

The  Benefits  and  Limitations  of  Self-Regulated  Learning  Microanalysis 
Scholars  and  practitioners  across  a  diverse  array  of  specialties  and  disciplines  have  long 
been  interested  in  understanding  why  individuals,  who  despite  possessing  adequate  intellectual 
and  cognitive  abilities  and  being  provided  with  sufficient  encouragement  and  feedback,  display 
inconsistent  performance.  The  answer  to  this  question  is  certainly  complex  and  necessitates 
consideration  of  a  range  of  non-intellectual  variables,  including  personality  characteristics, 
quality  of  instruction,  and  individual  beliefs,  attitudes,  and  strategic  skills.  Over  the  past  few 
decades,  researchers  across  different  domains  have  devoted  much  attention  to  examining  the  role 
that  self-regulated  learning  (SRL)  processes,  such  as  planning,  strategic  behaviors,  and 
reflection,  have  on  academic  learning  (Boekaerts,  Pintrich,  &  Zeidner,  2000;  Zimmerman  & 
Schunk,  2011),  mental  health  and  addictive  behaviors  (Baumeister  &  Vohs,  2007),  chronic 
disease  management  (Clark,  2013),  and  optimal  performance  in  sports  and  other  leisure  activities 
(Cleary,  Zimmerman,  &  Keating,  2006;  McPherson,  2011).  Despite  the  relatively  robust 
literature  linking  SRL  skills  with  success,  an  emergent  issue  in  the  SRL  literature  involves  the 
extent  to  which  different  assessment  methodologies  are  capable  of  capturing  unique  aspects  of 
how  individuals  regulate  during  authentic  tasks  and  activities  within  a  given  domain. 

In  this  paper,  we  provide  an  overview  of  one  type  of  assessment  methodology,  called 
SRL  microanalysis,  which  has  been  used  to  study  SRL  processes  across  a  variety  of  different 
tasks  and  activities.  We  detail  the  theoretical  foundation  of  this  assessment  methodology,  the 
core  assumptions  and  characteristics,  as  well  as  its  primary  strengths  and  weaknesses.  We  also 
provide  examples  of  how  this  methodology  has  been  used  by  researchers  to  examine  the  quality 
of  individuals’  regulatory  processes  during  academic  and  non-academic  tasks,  with  primary 
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emphasis  placed  on  a  recent  application  to  a  diagnostic  reasoning  task  in  a  medical  education 
context.  Before  we  discuss  SRL  microanalysis,  however,  we  first  describe  SRL  in  greater  detail 
and  present  a  theoretical  model  that  serves  as  the  foundation  for  SRL  microanalytic  procedures. 
Overview  of  SRL  as  a  Contextualized  Dynamic  Process 

SRL  has  been  described  as  a  process  through  which  individuals  exert  control  over  their 
behaviors,  cognitions  or  affect,  and  the  environment  as  they  engage  in  goal-directed  activities 
(Boekaerts  et  ah,  2000;  Schunk,  2012).  Although  we  recognize  that  there  are  several  different 
theories  of  SRL  and  that  researchers  across  various  disciplines  target  constructs  related  to  SRL, 
including  self-directed  learning,  self-management,  executive  functions,  and  reflective  practice 
(see  Cleary,  Durning,  Gruppen,  Hemmer,  &  Artino,  2013  for  an  overview),  we  focus  primarily 
on  social-cognitive  theoretical  principles  and  frameworks  (Bandura,  1986;  Zimmerman,  2000). 
On  a  general  level,  social-cognitive  theory  emphasizes  a  proactive,  agentic  perspective  of  human 
functioning  whereby  individuals  seek  out  appropriate  models  from  which  to  learn  and  use  their 
regulatory  capabilities  such  as  self-observation,  self-judgments,  and  self-reactions,  to  manage 
and  control  their  thoughts,  behaviors  and  enviromnent.  Social-cognitive  theorists  emphasize  that 
learning  and  behaviors  are  largely  contextualized  and  thus  what  an  individual  thinks  and  does 
will  often  vary  and  change  across  contexts. 

Building  from  this  initial  social-cognitive  framework,  Zimmerman  (2000)  developed  a 
model  of  SRL  that  emphasized  the  inter-relatedness  among  SRL  processes,  specifically  detailing 
how  the  different  sub-processes  operate  as  a  type  of  cyclical  feedback  loop.  From  this 
perspective,  SRL  has  been  defined  as  self-generated  thoughts,  actions,  and  feelings  that  are 
intentionally  and  proactively  generated  and  used  by  learners  to  attain  personal  goals 
(Zimmerman,  2000).  These  regulatory  processes  are  subsumed  within  a  three-phase  cyclical 
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feedback  loop,  consisting  of  three  general  phases:  forethought  (before  engaging  in  a  task), 
performance  (during  task  engagement),  and  self-reflection  (following  perfonnance  on  the  task). 
Forethought  phase  processes  include  establishing  personal  goals  and  strategic  plans  in  relation 
to  performing  specific  tasks.  Goal  setting  involves  deciding  upon  specific  outcomes  of  learning 
or  performance  while  strategic  planning  refers  to  the  selection  of  specific  strategies  that  will 
facilitate  the  acquisition  or  display  of  skill  (Zimmerman,  2000). 

As  sophisticated  self-regulated  learners  attempt  to  perform  a  task  or  activity  in  a  given 
situation  (i.e.,  during  the  performance  phase  of  the  model),  they  will  often  attempt  to  exert  self- 
control  over  personal  processes,  such  as  behavior,  affect,  and  cognition  (e.g.,  attention-focusing, 
self-instruction  statements)  as  well  as  the  environment  in  order  to  optimize  task  performance. 
Individuals  who  are  highly  regulated  also  tend  to  be  interested  in  gathering  data  about  their  skills 
and  performance  progress  and  thus  will  often  proactively  monitor  and  record  performance 
outcome  as  well  as  the  types  of  actions  and  thoughts  they  exhibit  during  learning  (i.e.,  self¬ 
observation).  Collectively,  these  strategic  and  monitoring-related  behaviors  are  purposefully 
selected  and  implemented  in  order  to  maximize  the  likelihood  of  successfully  performing  a  task 
and  to  gather  relevant  data  that  can  be  used  during  the  third  phase  of  the  model,  self-reflection. 

During  the  self-reflection  aspect  of  the  feedback  loop,  self-regulated  learners  will  often 
make  a  few  critical  self-judgments  about  their  performance,  such  as  whether  they  attained  their 
goals  (i.e.,  self-evaluation)  and  the  reasons  underlying  their  performance  (i.e.,  attribution).  These 
judgments  will,  in  turn,  impact  the  conclusions  or  inferences  that  learners  make  about  what  they 
need  to  do  to  improve  future  performance  on  a  particular  task  ( adaptive  inferences).  An  iteration 
of  the  cyclical  feedback  loop  is  complete  when  self-reflection  phase  processes  impact  students’ 
forethought  processes  prior  to  a  subsequent  learning  or  performance  activity.  This  process- 
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oriented,  dynamic  feedback  model  has  served  as  the  foundation  from  which  SRL  microanalytic 
methodology  has  evolved. 

Measurement  of  SRL 

Historically,  a  variety  of  assessment  approaches  have  been  used  to  study  SRL  behaviors 
and  processes.  The  most  common  method  involves  the  use  of  questionnaires  (Cleary,  2009; 
Dinsmore  et  ah,  2010),  whereby  individuals  are  asked  to  read  a  series  of  statements  about  their 
regulatory  behaviors  and/or  beliefs  and  attitudes  and  then  retrospectively  rate  to  what  extent 
those  statements  reflect  their  own  beliefs  and  behaviors  in  a  given  context.  Other  SRL 
assessments  include  teacher  or  parent  rating  scales  (Cleary  &  Callan,  in  press),  structured 
interviews  (Zimmerman  &  Martinez  Pons,  1986),  behavioral  traces  (Winne  &  Perry,  2000), 
think-aloud  protocols  (Azevedo,  2005),  direct  observations  (Perry,  1998),  and  SRL  microanalytic 
protocols  (Artino,  Cleary,  Dong,  Hernmer,  &  Durning,  2014;  Cleary,  Callan  &  Zimmerman, 
2012;  Cleary  &  Zimmerman,  2001).  Although  each  of  these  assessment  tools  has  a  unique 
profile  of  strengths  and  weaknesses,  an  emergent  trend  in  the  SRL  literature  has  been  to  utilize 
measures  that  have  the  following  characteristics:  (a)  highly  context-  and  task-specific,  (b) 
administered  in  real  time  during  authentic  tasks,  and  (c)  exhibit  strong  psychometric 
characteristics.  Examples  of  these  types  of  on-line  or  event  methods  include  think  alouds,  direct 
observations,  behavioral  traces,  and  SRL  microanalytic  protocols.  Unlike  self-report 
questionnaires  or  rating  scales,  event  measures  are  beneficial  because  they  are  not  adversely 
influenced  by  memory  distortions  or  errors  due  to  retrospective  responding.  Further,  because 
they  are  highly  contextualized  and  are  administered  in  relation  to  specific  tasks  and  activities,  the 
data  that  are  generated  have  the  potential  to  be  much  more  meaningful  for  practitioners.  In  this 
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paper,  we  devote  the  majority  of  our  attention  to  one  type  of  event  measure,  SRL  microanalytic 
protocols. 

Overview  of  SRL  microanalytic  methodology 

SRL  microanalysis  evolved  from  social-cognitive  theory  in  general  and  from 
Zimmerman’s  three-phase  cyclical  feedback  model  more  specifically.  This  approach,  which 
traditionally  has  utilized  a  context-specific,  structured  interview  protocol,  adheres  to  a  “strategic, 
coordinated  plan  of  administering  context-specific  questions  targeting  multiple  cyclical  phase 
sub-processes  as  students  engage  in  authentic  activities”  (Cleary  et  ah,  2012,  p.  4).  This 
assessment  approach  includes  the  following  essential  features:  (a)  individualizing  administration, 
(b)  using  theoretically-grounded,  highly  contextualized  questions,  (c)  linking  regulatory 
questions  to  the  phase  dimensions  of  the  three-phase  SRL  model,  and  (d)  coding  and  scoring 
responses.  In  this  paper,  we  focus  primarily  on  the  nature  of  microanalytic  questions  and  how 
they  are  administered  during  the  interview  session. 

Nature  of  microanalytic  questions.  All  microanalytic  questions  are  developed  based  on 
theoretical  definitions  of  the  phase-specific  SRL  sub-processes.  Typically,  researchers  have 
evaluated  several  self-regulatory  processes  (e.g.,  goal  setting,  strategic  planning,  self-efficacy, 
attributions)  within  and  across  the  three  SRL  cyclical  phases.  Although  the  microanalytic 
questions  can  either  be  open-  or  closed-ended,  the  majority  of  questions  utilize  a  free-response 
format  that  enables  individuals  to  provide  elaborated  responses  (Cleary  et  al.,  2012;  Cleary  & 
Zimmerman,  2001).  From  our  perspective,  a  free-response  format  is  ideal  because,  unlike  self- 
report  questionnaires,  it  does  not  provide  the  “answers”  to  which  individuals  simply  have  to 
make  a  rating  or  judgment.  Rather,  respondents  are  required  to  provide  their  thoughts,  beliefs, 
and  reactions  in  a  particular  moment  in  time  as  they  complete  a  specific  task.  Given  the  highly 
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contextualized  and  task-focused  nature  of  this  approach,  single  questions  are  typically  used  to 
target  each  of  the  specific  SRL  sub-processes  subsumed  within  the  three-phase  loop.  There  is  a 
strong  literature  base  showing  that  these  single-item,  contextualized  questions  have  high  levels 
of  inter-rater  reliability,  can  adequately  distinguish  between  high  and  low  achievers,  and  can 
account  for  unique  variance  in  future  performance  outcomes  (Artino  et  ah,  2014;  Cleary  et  ah, 
2012;  DiBenedetto  &  Zimmerman,  2013;  Kitsantas  &  Zimmerman,  2002). 

Linking  microanalysis  to  temporal  task  dimensions.  Each  of  the  SRL  microanalytic 
questions  are  purposefully  developed  and  administered  to  examine  individuals’  SRL  processes  as 
they  engage  in  authentic  tasks  and  activities  (Cleary  et  al.,  2012).  To  date,  these  protocols  have 
been  developed  and  customized  to  a  variety  of  domains,  including  athletic  skills  (e.g.,  free-throw 
shooting,  volleyball  serving;  Cleary  &  Zimmerman,  2001;  Kitsantas  &  Zimmerman,  2002), 
academic  tasks  (e.g.,  reading,  studying,  test  preparation;  Cleary,  Callan,  Peterson,  &  Adams, 
2011;  DiBenedetto  &  Zimmerman,  2013;  Zimmerman  &  Kitsantas,  1999),  and  clinical  activities 
in  medicine  (e.g.,  venepuncture,  diagnostic  reasoning;  Artino  et  al.,  2014;  Cleary  &  Sandars, 
2011;  see  Table  1  for  an  abbreviated  list  of  studies  using  microanalysis  across  various  tasks). 

Regardless  of  the  specific  task  around  which  one  seeks  to  embed  the  SRL  microanalytic 
protocol,  it  is  important  that  the  task  has  a  well-defined  and  clear  beginning,  middle  and  end. 
Clarifying  the  temporal  dimensions  of  a  task  is  essential  because  it  allows  one  to  directly  embed 
the  forethought,  performance,  and  reflection  questions  relative  to  these  specific  task  dimensions. 
In  essence,  the  microanalytic  protocol  is  structured  so  that  the  forethought  phase  questions  are 
administered  before  one  begins  the  task,  performance  phase  questions  are  administered  during 
the  task,  and  reflection  phase  questions  are  administered  after  performance.  Establishing  direct 
links  between  SRL  theory,  the  assessment  process,  and  the  nature  of  a  task  allows  researchers  or 


practitioners  to  make  theoretically  grounded  interpretations  about  how  SRL  process  are  initiated, 
sustained,  or  modified  during  a  relevant  task. 

Advantages  and  limitations  of  SRL  microanalysis 

In  school  contexts,  legal  mandates  and  best  practices  require  that  diagnosticians  or  school 
psychologists  conduct  comprehensive  evaluations  that  involve  using  a  variety  of  assessment 
tools  (e.g.,  standardized  tests,  questionnaires,  interviews)  administered  to  different  sources  (e.g., 
students,  teachers)  in  order  to  gather  data  about  different  areas  of  functioning  (e.g.,  intellectual, 
academic)  for  a  student  (Cleary  et  ah,  2012).  The  basic  assumption  of  this  practice  is  that  a  single 
measure  can  rarely  capture  all  the  components  of  a  construct,  particularly  when  a  construct,  like 
self-regulation,  is  multi-dimensional  and  dynamic  in  nature.  Further,  despite  many  assessment 
tools  sharing  specific  characteristics  or  features,  most  have  a  unique  set  of  strengths  and 
weaknesses  that  make  them  desirable  under  certain  conditions  but  less  effective  in  other 
contexts.  Although  the  use  of  a  multi-dimensional  assessment  approach  is  a  goal  of  SRL 
measurement  (Butler,  2011;  Cleary  &  Platten,  2013),  in  this  section  we  underscore  the  primary 
advantages  and  limitation  of  SRL  microanalysis,  particularly  when  viewed  in  relation  to  other 
types  of  SRL  measures. 

Strengths  and  advantages.  Unlike  many  types  of  SRL  measures  that  rely  on 
retrospective,  aggregated  ratings  of  items  that  are  often  devoid  of  focused  situational  referents, 
such  as  self-report  questionnaires  or  teacher  ratings,  SRL  microanalytic  protocols  provide  a  more 
contextually  rich,  in-the-moment  profile  of  SRL  processes  that  relate  to  particular  tasks  that  are 
deemed  to  be  important  or  relevant  in  a  given  domain.  From  a  diagnostic  point  of  view,  because 
microanalytic  protocols  are  highly  contextualized  and  task-specific,  the  data  that  they  generate 
have  strong  potential  to  help  educators  and  practitioners  identify  deficient  regulatory  processes 


and  to  use  those  data  as  a  platform  from  which  to  intervene  with  students  who  are  struggling 
(Cleary  &  Platten  2013;  Peters-Burton,  2013). 
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As  indicated  previously,  SRL  microanalysis  is  a  type  of  event  measure  because  it  targets 
regulatory  skills  as  they  occur  during  an  event  or  specific  task.  In  this  respect,  they  are  similar  to 
other  measures  such  as  behavioral  traces,  observations,  and  think  alouds.  However,  SRL 
microanalytic  measures  are  distinct  from  behavioral  traces  and  observations  because 
microanalytic  protocols  are  designed  to  measure  covert  SRL  processes  including  students’  self- 
efficacy  perceptions,  goals,  plans,  and  reflective  judgments,  such  as  attributions.  Given  that 
social-cognitive  models  of  SRL  assume  that  thoughts,  beliefs,  and  metacognition  directly  impact 
behaviors,  it  is  important  to  gather  information  about  these  covert  processes  to  more  clearly 
understand  the  nature  of  the  behaviors  that  may  be  observed. 

Think-aloud  protocols  and  SRL  microanalysis  procedures  are  similar  in  that  they  involve 
gathering  information  about  an  individual’s  strategic  thoughts  and  beliefs  as  he  or  she  perfonns 
an  activity.  They  are  distinct,  however,  in  the  procedures  used  to  elicit  the  person’s  thoughts  and 
verbalizations.  Think  alouds  entail  having  individuals  talk  aloud  as  they  perform  a  task,  without 
the  examiner  providing  any  prompts  or  asking  any  questions.  In  contrast,  microanalytic  measures 
purposefully  and  strategically  present  questions  to  individuals  about  their  SRL  processes  at 
different  points  during  task  completion.  Although  each  of  these  two  approaches  is  beneficial  and 
important,  the  advantage  of  SRL  microanalytic  protocols  is  that  they  can  capture  specific  types 
of  information  that  can  be  used  to  draw  conclusions,  specifically  about  how  individuals  prepare, 
self-direct,  and  reflect  on  task  performance. 

Another  strength  of  microanalytic  methodology  is  the  emerging  evidence  that  this  type  of 
assessment  possesses  strong  psychometric  properties,  as  evidenced  by  inter-rater  reliability, 
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differential  validity,  convergent  validity,  and  predictive  validity  (Cleary  et  al.,  2012). 
Furthermore,  there  is  emerging  evidence  that  SRL  microanalytic  protocols  are  more  predictive  of 
performance  outcomes  that  other  forms  of  SRL  assessment,  such  as  self-report  questionnaires 
(Callan  &  Cleary,  2014;  Cleary  et  al.,  2011). 

Weaknesses  and  limitations.  As  is  the  case  with  most  assessment  approaches,  SRL 
microanalytic  protocols  do  possess  some  limitations.  For  example,  SRL  microanalytic  questions 
are  highly  contextualized  and  target  individuals’  regulatory  processes  during  a  single  moment  in 
time  as  they  perform  a  particular  task.  Although  this  feature  is  desirable  from  a  contextualist 
point  of  view,  the  data  gathered  from  this  assessment  approach  may  not  generalize  to  other 
contexts  or  situations.  Thus,  this  approach  was  developed  to  be  primarily  concerned  with 
addressing  the  question,  “How  is  the  person  regulating  their  behaviors  during  this  particular 
event  or  activity?”  rather  than,  “How  does  this  person  regulate  their  behaviors  in  general?“ 
Another  weakness  is  that  because  SRL  microanalytic  protocols  entail  an  examiner  asking 
questions  of  a  respondent  during  a  task,  one  can  question  whether  the  responses  are  true  accounts 
of  what  individuals  would  have  been  thinking  if  the  microanalytic  probe  or  prompt  had  not  been 
administered  (i.e.,  reactivity  effects). 

A  third  potential  drawback  of  microanalytic  protocols  is  that  because  they  historically 
have  been  primarily  concerned  with  measuring  covert  aspects  of  SRL,  such  as  thought  processes 
and  metacognitive  judgments  and  reactions,  they  have  not  typically  been  used  to  directly 
measured  overt  indicators  of  SRL,  such  as  help-seeking  behavior  and  actual  use  of  strategies. 
From  our  perspective,  however,  the  term  SRL  microanalysis  does  not  restrict  assessment  to  only 
the  cognitive  and  metacognitive  dimensions.  In  fact,  researchers  have  used  various  assessment 
approaches  that  track  micro-level  behaviors,  such  as  mother-infant  interactions  (Peck,  2003), 
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interactions  among  multiple  family  subsystems  (Gordon  &  Feldman,  2008),  and  therapist-client 
interactions  (Stiles  et  al.,  2006).  Although  not  systematically  examined  at  this  point,  it  is  critical 
for  microanalytic  researchers  to  begin  to  examine  how  traditional  microanalytic  questions  can  be 
supplemented  with  microanalytic,  behavioral  indicators  of  those  covert  processes. 

Finally,  although  recent  attempts  have  been  made  to  apply  SRL  microanalysis  to  examine 
cognitive  activities,  such  as  clinical  reasoning  with  medical  students,  it  is  important  to 
underscore  that  microanalytic  protocols  are  not  well-suited  to  examine  automatic  or  unconscious 
processes  that  may  occur  during  reasoning  activities,  particularly  in  those  who  are  expert 
clinicians  (Norman,  2005). 

Applications  of  SRL  microanalysis 

As  alluded  to  previously,  SRL  microanalytic  protocols  have  been  extended  and  applied  to 
a  diverse  array  of  tasks  (see  Table  1).  In  addition  to  the  broad  applicability  of  this  assessment 
paradigm,  researchers  have  used  this  approach  for  a  variety  of  different  purposes  and  objectives, 
including  identifying  group  differences  (Cleary  &  Zimmerman,  2001;  DiBendetto  & 

Zimmerman,  2013),  predicting  performance  outcomes  (Artino  et  al.,  2014;  Kitsantas  & 
Zimmerman,  2002),  evaluating  the  effects  of  interventions  (Cleary  et  al.,  2006),  and  most 
recently,  exploring  shifts  in  students’  regulatory  processes  during  a  task  (Cleary,  Dong,  & 

Artino,  2014).  Collectively,  various  ex  post  facto  studies  have  shown  that  experts  or  high 
achievers  display  more  adaptive  fonns  of  SRL,  such  as  setting  more  specific  goals  and  exhibiting 
adaptive,  strategic  forms  of  planning  and  self-reflection.  For  example,  Cleary  &  Zimmerman 
(2001)  showed  that  expert  basketball  free-throw  shooters  in  high  school  displayed  more  specific 
goals  and  higher  quality  strategic  plans  and  attributions  than  their  non-expert  teammates  and 
novices.  There  is  also  emerging  data  that  SRL  processes  not  only  predict  proximal  and  task- 
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specific  outcomes  but  also  long-term  perfonnance  indicators.  For  example,  whereas  Kitsantas 
and  Zimmerman  (2002)  showed  that  microanalytic  responses  of  volleyball  players  predicted 
approximately  90%  of  the  variance  in  the  target  task  of  volleyball  serving,  Artino  and  colleagues 
(2014)  found  that  a  strategic  planning  microanalytic  question  accounted  for  approximately  8- 
14%  of  the  variance  in  important  medical  school  perfonnance  indicators  (e.g.,  licensing 
examination  scores)  after  controlling  for  reasoning  ability  and  prior  achievement. 

Other  lines  of  research  have  examined  changes  in  individuals’  SRL  microanalytic 
processes  following  an  intervention.  In  an  experimental  study  with  college-aged  students,  Cleary 
et  al.  (2006)  showed  that  novice  basketball  players  who  received  multiple -phase  training  in  SRL 
processes  (i.e.,  training  in  forethought,  perfonnance,  and/or  self-reflection)  as  they  practiced 
their  free-throw  shooting,  exhibited  more  adaptive  attributions  and  adaptive  inferences  following 
missed  free-throws  and  an  overall  stronger  shooting  skill  than  those  who  did  not  receive  such 
training.  In  the  following  section,  we  review  a  recent  study  that  used  SRL  microanalysis  to 
examine  shifts  in  students’  SRL  processes  during  a  diagnostic  reasoning  activity  (Cleary,  Dong, 
&  Artino,  2014).  Our  primary  objective  is  to  illustrate  how  microanalysis  was  employed  to 
investigate  SRL  processes  during  multiple  iterations  of  this  task. 

Illustration  of  the  SRL  microanalytic  process.  Cleary  et  al.  (2014)  was  one  of  the  first 
studies  to  employ  microanalytic  methodology  to  examine  how  medical  students’  self-efficacy 
and  quality  of  strategic  thinking  (i.e.,  planning,  metacognitive  monitoring)  shifted  as  they 
completed  multiple  iterations  of  a  diagnostic  reasoning  activity.  In  this  study,  7 1  second-year 
medical  students  were  asked  to  participate  in  a  25-30  minute  diagnostic  reasoning  session. 

During  this  session,  read  a  one-page  paper  case  depicting  diabetes  mellitus  and  were  afforded  the 
opportunity  to  use  a  post-encounter  form  (PEF)  as  a  guide  for  developing  and  generating  an 
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accurate  diagnosis.  After  submitting  their  final  diagnosis,  the  students  were  provided  simple 
corrective  feedback:  “Sorry,  your  most  likely  diagnosis  is  incorrect.”  The  participants  were  then 
given  the  opportunity  to  complete  another  PEF  as  they  engaged  in  the  second  iteration  of  the 
same  clinical  reasoning  activity.  Following  their  second  attempt  at  generating  a  most  likely 
diagnosis,  the  students  were  again  read  a  similar  corrective  feedback  statement.  It  should  be 
noted  that  the  diagnostic  reasoning  task  was  intentionally  developed  to  be  challenging  because 
the  authors  were  interested  in  examining  students’  reactions  following  feedback  about  their 
failure  to  arrive  at  a  correct  diagnosis. 

Nature  of  microanalytic  questions.  To  evaluate  shifts  in  students’  self-efficacy  beliefs  and 
SRF  processes  during  the  multiple  iteration  activity,  the  authors  embedded  three  microanalytic 
questions  into  the  diagnostic  reasoning  task:  self-efficacy,  strategic  planning,  and  metacognitive 
monitoring.  The  single  item  self-efficacy  measure  was  developed  using  guidelines  outlined  by 
Bandura  (2006)  and  examined  the  participants’  confidence  about  their  ability  to  generate  the 
correct  diagnosis  (see  Figure  1).  A  single-item  measure  of  strategic  planning  was  used  to  assess 
participant  plans  for  approaching  the  diagnostic  reasoning  task.  A  variation  of  an  existing 
planning  question  was  used  (Cleary  et  ah,  2012).  Participants  were  asked,  “What  do  you  think 
you  need  to  do  to  perform  well  on  this  activity?”  Participant  responses  to  the  strategic  planning 
question  were  coded  into  one  of  six  categories:  task-specific  process,  task-general  process,  self- 
control,  non-task  strategy,  do  not  know/none,  and  other.  Finally,  the  metacognitive  monitoring 
question  examined  the  extent  to  which  the  participants  focused  on  strategic  processes  during 
each  of  the  two  iterations  of  the  clinical  reasoning  task.  Across  both  iterations,  the  participants 
were  stopped  after  they  wrote  down  their  first  differential  diagnosis  on  the  PEF  and  asked,  “As 
you  have  been  going  through  this  process,  what  has  been  the  primary  thing  you  have  been 
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thinking  about  or  focusing  on?”  If  the  participants  provided  a  response,  they  were  asked,  “Is 
there  anything  else  you  have  been  focusing  on?”  Student  responses  were  coded  into  one  of  seven 
categories.  Four  categories  were  similar  to  the  strategic  planning  measure,  task-specific  process, 
task-general  process,  self-control,  and  other.  Three  additional  categories  were  added  to  this 
coding  scheme,  outcome,  perceived  ability  and  task  difficulty.  Using  similar  coding  procedures 
employed  for  the  strategic  planning  measure,  an  inter-rater  level  of  agreement  of  88%  was 
attained  for  both  strategic  planning  and  metacognitive  monitoring  questions. 

Given  that  the  strategic  planning  and  metacognitive  monitoring  measures  used  a  firee- 
response  format,  and  because  participants  were  permitted  to  provide  multiple,  codeable 
responses  to  each  question,  the  authors  transfonned  the  categorical  responses  to  a  metric  scale  to 
facilitate  interpretation.  The  scoring  system  was  designed  to  capture  the  strategic  quality  of  the 
participants’  regulatory  processes  during  the  specific  task,  with  greatest  weight  given  to 
responses  that  reflected  one  or  more  of  the  five  strategic  steps  identified  for  the  diagnostic 
reasoning  task.  This  scoring  system  was  an  adaptation  of  a  prior  scoring  scheme  and  was 
developed  based  on  theory,  prior  research,  and  expert  consensus  (Artino  et  ah,  2014;  Cleary  et 
al„  2011). 

Link  between  measures  and  diagnostic  reasoning  task.  Consistent  with  the  guidelines  put 
forth  by  Cleary  (2011),  there  was  a  direct  link  between  the  temporal  dimensions  of  the  diagnostic 
reasoning  task  and  the  administration  sequence  of  microanalytic  questions  (see  Figure  1).  For 
example,  the  microanalytic  questions  were  purposefully  administered  to  parallel  the  temporal 
dimensions  of  the  task,  such  that  self-efficacy  and  strategic  planning  (i.e.,  forethought  phase 
processes)  were  administered  prior  to  attempting  each  of  the  two  iterations.  These  two 
forethought  phase  processes  were  assessed  at  three  points:  prior  to  initiating  the  task,  before 
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beginning  the  second  iteration  of  the  task,  and  prior  to  a  prospective  third  iteration.  Given  that 
the  authors  were  also  interested  in  gathering  information  about  the  types  of  thoughts  that  students 
were  exhibiting  during  both  iterations  of  the  diagnostic  reasoning  task,  they  administered  a 
metacognitive  monitoring  probe  (which  is  a  performance  phase  process)  during  both  task 
iterations. 

Results  and  implications.  Repeated  measure  analyses  were  used  to  examine  the  presence 
of  linear  or  quadratic  trends  across  three  time  points  (before  the  first  iteration,  before  the  second 
iteration,  and  before  a  prospective  third  iteration)  for  the  self-efficacy  and  strategic  planning 
measures  (see  Figure  1).  Separate  paired  t-tests  were  used  to  examine  shifts  from  Time  1  to  Time 
2  and  from  Time  2  to  Time  3.  Finally,  a  simple  paired  t-test  was  used  to  examine  changes  in  the 
metacognitive  monitoring  question,  given  that  only  two  data  points  were  collected.  See  Table  2 
for  an  overview  of  means  and  standard  deviations  of  measure  across  time  points. 

The  key  result  was  that  although  the  participants’  exhibited,  on  average,  a  moderate 
levels  of  self-efficacy  and  focused  on  the  strategic  qualities  of  the  diagnostic  reasoning  task  at 
the  beginning  of  the  sessions,  they  exhibited  statistically  significant  declines  in  both  the 
confidence  levels  {self-efficacy,  7^(1 ,67)  =  1 14.37,  p  <  .001,  rjp=  .631)  and  the  quality  of  their 
strategic  thinking  {strategic planning-,  F(l,70)  =  5.32 ,p  =  .024,  symbol  rjp  =  .07;  metacognitive 
monitoring,  t(70)  =  2.91, p  =  .005,  <7  =  0.49  )  during  the  20-30  minute  activity.  Qualitatively,  the 
results  showed  that  although  approximately  two  thirds  of  the  participants  (65%)  focused  on  the 
strategic  process  typically  used  in  diagnostic  reasoning  (e.g.,  identify  symptoms,  integration  etc.) 
at  the  beginning  of  the  task,  less  than  25%  of  participants  sustained  this  focus  on  strategic 
thinking  at  the  end  of  the  task.  The  declines  in  strategic  thinking  paralleled  the  large  drops  in 
participant  self-efficacy  during  the  task. 
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From  our  perspective,  the  most  striking  aspect  of  this  study  entailed  the  immediacy  and 
size  of  the  drop  in  both  self-efficacy  and  quality  of  strategic  thinking  over  a  relatively  short  time 
period.  These  findings  are  particularly  relevant  to  medical  educators  because  clinical  reasoning 
activities  are  a  critical  focus  in  medical  education  and  because  the  practice  session  used  in  this 
study  represented  a  close  approximation  of  a  typical  practice  opportunity  provided  to  students 
enrolled  in  a  medical  education  course  at  the  target  school.  It  is  important  for  medical  educators 
to  be  cognizant  of  how  quickly  student  motivation  and  thinking  can  change  during  a  single  task 
and  how  providing  simple,  negative  task  feedback  can  push  some  students  toward  a  maladaptive 
path  of  self-doubt  and  potential  withdrawal  or  disengagement  (Bandura,  1997;  Hattie  & 
Timperley,  2007;  Pajares  &  Urdan,  2006). 

Where  does  SRL  microanalysis  go  from  here? 

One  of  the  original  purposes  of  developing  SRL  microanalytic  protocols  was  to  provide 
diagnostic  information  that  practitioners,  clinicians,  or  educators  can  use  to  provide  process  or 
SRL-type  feedback  to  students  about  how  they  approach  and  go  about  completing  important 
tasks  (Cleary  et  ah,  2012).  At  this  point,  although  a  few  researchers  have  begun  to  provide 
qualitative  and  anecdotal  evidence  for  the  viability  of  using  SRL  microanalytic  data  for  this 
purpose  (Cleary  &  Platten,  2013;  Peters-Burton,  2013),  there  is  a  need  to  more  systematically 
examine  this  issue  in  a  more  comprehensive  fashion.  In  particular,  it  is  important  to  consider 
how  professional  educators  and  trainers,  who  may  not  be  experts  in  SRL  terminology  and 
principles,  can  use  this  approach  in  their  work  and  interactions  with  students. 

We  also  envision  integrating  SRL  microanalytic  protocols  with  behavioral  observations 
and  behavioral  traces  of  SRL  and  motivated  behaviors.  Although  there  is  ample  evidence  to 
support  the  use  of  SRL  microanalytic  protocols  to  differentiate  achievement  groups  across 
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empirically  supported  constructs  (e.g.,  goal  setting,  planning  etc.),  greater  attention  should  be 
devoted  to  exploring  how  SRL  microanalytic  data  converge  with  behavioral  indicators  of  SRL 
and  to  explore  the  reciprocal  nature  of  the  relation  between  these  two  sources  of  data. 

In  this  paper,  we  provided  an  overview  of  SRL  microanalysis,  detailing  the  theoretical 
foundation  of  this  assessment  methodology,  the  core  assumptions  and  characteristics,  and  its 
primary  strengths  and  weaknesses.  We  also  provided  examples  of  how  this  methodology  has 
been  used  by  researchers  to  examine  the  quality  of  individuals’  regulatory  processes  during 
academic  and  non-academic  tasks,  including  specific  emphasis  on  a  recent  application  to 
diagnostic  reasoning  tasks  in  medical  education.  Although  much  more  research  is  clearly  needed 
to  examine  the  utility  and  psychometric  quality  of  this  assessment  approach,  particularly  as  it 
relates  to  different  populations  and  contexts,  we  believe  SRL  microanalysis  has  the  potential  to 
improve  the  quality  with  which  educators  provide  instruction,  feedback,  and  remediation  to 


students  in  various  educational  contexts. 
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Table  1 

Selected  examples  of  SRL  Microanalytic  Studies  Across  Tasks  and  Domains 


Type  of  Task 

Study 

Forethought 

Performance 

Self-reflection 

Athletic  tasks 

Dart  throwing 

Kitsantas, 
Zimmerman,  & 
Cleary,  (2000) 

-  Attribution 

Basketball 

Cleary  & 

-  Goal  setting 

- 

-  Attribution 

shooting 

Zimmerman 

(2001) 

-  Strategy  choice 

-  Adaptive  infer. 

Volley  Ball 

Kitsantas  & 

-  Goal  setting 

-  Strategy  use 

-  Self-evaluation 

serving 

Zimmerman 

-  Planning 

-  Self-monitor 

-  Attribution 

(2002) 

-  Strategic  plan 

Academic  tasks 

Reading  and 

Dibenedetto  & 

-  Goal  setting 

-  Metacognitive 

-  Self-evaluation 

studying  in 

Zimmerman 

-  Strategic  plan 

monitoiring 

-  Attribution 

science 

(2010) 

-  Strategy  use 

-  Adaptive  infer. 

-  Satisfaction 

Test  reflection 

Cleary,  Callan, 

-  Strategic  plan 

- 

-  Self-evaluation 

Peterson,  & 

-  Attribution 

Adams  (2011) 

-  Adaptive  infer. 

Math  problem¬ 

Callan  &  Cleary 

-  Goal  setting 

-  Strategy  use 

-  Attributions 

solving 

(2014) 

-  Strategic  Plan 

-  Metacognitive 
monitoiring 

-  Adaptive  infer. 

Clinical  tasks 

Venepuncture 

Cleary  and 

-  Goal  setting 

-  Metacognitive 

-  Self-evaluation 

Sandars  (2011) 

-  Strategic  plan 

monitoiring 

(forced  choice) 

Diagnostic 

Artino  et  al 

-  Goal  setting 

-  Metacognitive 

reasoning  task 

(2014) 

-  Strategic  plan 

monitoiring 

Diagnostic 

Cleary 

-  Strategic  plan 

-  Metacognitive 

- 

reasoning 

etal.,  (2014) 

monitoiring 
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Table  2.  Means  and  standard  deviations  for  all  dependent  measures 


Timing  of 
assessment 

Self-efficacy 

Strategic  planning 

Metacognitive 

monitoring 

Mean 

SD 

Mean 

SD 

Mean 

SD 

Time  1 

53.73 

18.51 

.86 

1.74 

3.08 

2.05 

Time  2 

44.30 

18.43 

1.15 

1.74 

1.94 

2.57 

Time  3 

27.13 

17.48 

.10 

1.87 

NA 

NA 

Note.  For  the  self-efficacy  and  planning  measures,  Time  1  =  before  first  iteration;  Time  2  = 
before  second  iteration;  and  Time  3  =  before  prospective  third  iteration.  The  metacognitive 
monitoring  measure  was  only  administered  twice  during  the  task,  Time  1  =  during  first  iteration 
whereas  Time  2  =  during  second  iteration.  NA  =  Not  applicable. 
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Figure  1.  To  evaluate  shifts  in  students’  SRL  processes  and  self-efficacy  beliefs  during  the  multiple  iteration  activity,  an  SRL 
microanalytic  interview  was  administered  to  the  participants  at  different  points  during  the  task.  The  form  and  sequence  of  each 
interview  question  is  provided  here. 


